Co-operation through Hierarchical Competition in Genetic Data Mining

Data mining is the automatic search for interesting and useful correlations within large databases. It lays particular emphasis on discovering patterns that are hard or impossible to detect using standard query mechanisms and classical statistical techniques. This paper describes a formulation of data mining as a well-defined search problem and shows how a hierarchical implementation of a genetic algorithm can be used to execute this task. Such a data miner has been implemented using RPL2, a language and parallel framework for evolutionary computation. The resulting system has been used to search a retail sales database from a major high-street chain, successfully producing interesting characterisations of consumer behaviour. The hierarchical genetic algorithm introduced is of relevance not only to data mining, but also to general covering problems.